Presentation 


Potential  Payoff  of  Fusion  between  HSI  and  other  Sensors^ 

Su  May  Hsu^  and  Hsiao-hua  Burke 
MIT  Lincoln  Laboratory 
244  Wood  St.  Lexington,  Mass.  02420-9108 


Abstract 


In  this  paper,  two  examples  of  sensor  fusion  are  demonstrated.  The  first  is  hyperspectral  imaging  (HSI) 
with  Synthetic  Aperture  Radar  (SAR)  and  the  other  is  HSI  with  high-resolution  panchromatic  imaging 
(HPI).  HSI  and  SAR  fusion  exploits  different  phenomenologies  from  distinctly  different  sensors.  HSI  and 
HPI  fusion  optimizes  their  superior  respective  spectral  and  spatial  information.  Examples  are  illustrated 
using  field  measurements  as  well  as  simulated  data.  Fusion  of  SAR  and  HSI  data  is  established  at  feature 
level.  The  fusion  results  showed  reduced  false  detection  and  confirmed  target  detection  in  the  SAR  image 
with  background  characterization  and  material  identification  from  HSI.  Fusion  of  HSI  and  HPI  is  shown  at 
both  data  and  feature  levels.  The  combined  high-resolution  spatial-spectral  analysis  is  illustrated  for 
enhanced  target  identification. 

Key  words:  Hyperspectral  imaging  (HSI),  Synthetic  Aperture  Radar  (SAR),  high-resolution  panchromatic 
imaging  (HPI),  principal  component  analysis  (PCA),  background  classification,  matched  filtering,  image 
sharpening,  edge  detection. 

1  Introduction 

Hyperspectral  imaging  sensors  collect  data  in  a  three-dimensional  image  cube.  It  is  resolved  in  along-track, 
cross-track  and  spectral  dimensions.  It  has  fine  spectral  resolutions  (A^  ~  10  nm),  typically  in  visible  to 
SWIR  wavelength  region  (0.4  -  2.5  |im).  For  each  ground  sample  distance  (GSD)  pixel  within  a 
hyperspectral  image,  a  continuous  spectrum  is  measured  and  used  to  identify  materials  by  their  reflectance. 
But  it  provides  no  surface  penetrations.  Additionally,  to  maintain  required  SNR,  HSI  spatial  resolutions  are 
in  general  coarser  than  broadband  imagery  resolutions.  Thus  there  is  great  potential  to  enhance  the  overall 
HSI  system  performance  if  fusion  with  other  remotely  sensed  data  can  be  accomplished.  For  example,  in 
the  application  of  counter  camouflage,  concealment  and  deception  (CC&D),  HSI  can  be  used  to  identify 
ground  coverage  and  surface  material.  And  a  FOPEN  SAR  (Foliage  Penetration  Synthetic  Aperture  Radar) 
can  determine  if  any  threat  objects  are  under  concealment.  In  the  surface  surveillance  application,  HSI  can 
be  augmented  with  high-resolution  broadband  panchromatic  images  to  achieve  simultaneous  high  spectral 
and  spatial  resolutions  for  enhanced  target  detection  and  identification. 

There  have  been  several  examples  in  which  both  Synthetic  Aperture  Radar  (SAR)  and  Hyperspectral 
Imaging  (HSI)  systems  collected  data  in  support  of  military  operations.  Principles  of  low  frequency  SAR 
and  Hyperspectral  imaging  are  different  and  their  detection  capabilities  often  compliment  each  other. 
FOPEN  SAR  operates  at  20-700  MHz.  It  penetrates  foliage  and  detects  targets  under  tree  canopy,  but  has 
significant  clutter  returns  from  trees.  Hyperspectral  imaging,  on  the  other  hand,  is  capable  of  sub-pixel 
detection  and  material  identification.  Both  SAR  and  HSI  systems  may  suffer  substantial  false  alarm  and 
leakage  rates  due  to  respective  background  clutter.  It  is  expected  that  a  combined  SAR  and  HSI  system 
will  greatly  enhance  the  detection  and  identification  performance.  It  is  possible  that  a  combined  SAR/HSI 
system  could  have  the  imaging  sensors  cue  each  other  in  joint  observations.  Potential  applications  for  such 
fusion  include  counter  camouflage,  concealment  and  deception  (CC&D)  and  surveillance  of  exposed  sites. 

Hyperspectral  imaging  sensors  collect  image  data  in  hundreds  of  contiguous  narrow  spectral  bands  (~10 
nm)  with  moderate  spatial  resolutions.  In  contrast,  conventional  single  band  or  multi-spectral  sensors 
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typically  image  in  wide,  discrete  spectral  bands  with  fine  spatial  resolutions.  If  hyperspectral  image  can  be 
spatially  “sharpened”  via  the  broadband  high-resolution  image,  the  detection  and  identification 
performance  is  expected  to  improve  further.  Currently,  there  are  a  number  of  space-based  hyperspectral 
measurement  platforms  to  be  launched  in  the  foreseeable  future:  NASA’s  EO-1,  Air  Force’s  Warfighter-1 
and  Navy’s  NEMO  (Naval  Earth  Map  Observer).  Onboard  these  systems  there  are  high-resolution 
panchromatic  imaging  (HPI)  sensors  in  addition  to  HSI  sensors,  with  the  linear  spatial  resolution  of  3  to  8 
times  better.  It  is  important  that  effective  approaches  of  hyperspectral  image  fusion  with  the  high- 
resolution  broad-band  image  be  explored  to  enhance  the  utility  of  future  HSI  data. 

In  the  next  section,  an  example  of  HSI/SAR  fusion  is  demonstrated.  Since  HSI  and  SAR  are  distinctly 
different  sensors  exploiting  different  phenomenology,  fusion  of  HSI  and  SAR  data  is  established  at  feature 
level.  In  Section  3,  fusion  of  HSI/HPI  is  explored.  Sharpening  of  HSI  with  an  HPI  is  first  investigated. 
Then  a  combined  spatial-spectral  analysis  is  illustrated  for  enhanced  target  detection  and  identification 

2  HSI/SAR  Fusion 

A  common  data  set  from  Dixie-97  data  collect  over  Vicksburg,  Mississippi  is  used  here  to  demonstrate  the 
framework  of  SAR/HSI  fusion.  Both  P-3  UWB  radar  (215-730  MHz)  and  HYDICE  (Hyperspectral  Digital 
Image  Collection  Experiment,  0.4-2. 5  jim  in  210  bands)  collected  data  at  the  site.  A  sketch  of  the  target 
site  is  shown  in  Figure  1.  Targets  in  the  forest  background  included  fabric  nets  and  vehicles.  Several  fabric 
nets  were  populated  along  the  tree  line  around  an  open  area.  One  fabric  net  at  the  tree  line  was  covering  a 
vehicle.  All  other  nets  were  either  empty  or  covering  non-radar  reflecting  decoys.  In  addition,  there  was 
one  vehicle  obscured  at  the  lower  right  corner  of  the  sketch  and  another  vehicle  was  partially  exposed  near 
the  top  left  comer.  The  SAR  data  was  collected  with  a  32°  depression  angle  and  a  ground  sample  distance 
(GSD)  of  0.23m  x  0.4m.  The  HSI  data  was  collected  at  1 .5  km  altitude  with  a  nadir  viewing  geometry  and 
GSD  of0.76mx  1.1m. 

For  the  Dixie  data  collect,  a  SAR/HSI  fusion  strategy  is  established  based  on  their  detection  characteristics. 
SAR  and  HSI  data  are  first  processed  separately  for  detection  and  terrain  classification,  respectively.  Then 
co-registration  is  performed  to  allow  overlay  of  the  images.  SAR  false  alarm  from  trees  are  reduced  by 
terrain  mapping,  detection  of  concealed  targets  under  nets  is  verified  and  detection  of  partially  exposed 
targets  is  further  confirmed  with  material  identification  by  HSI.  In  this  section,  analysis  of  the  HSI  data  is 
first  given,  followed  by  illustration  of  SAR/HSI  image  co-registration  and  the  fusion  results. 


Figure  1  Common  data  set  of  SAR/HSI 
from  Dixie  data  collect,  May  1997 


2. 1  HSI  Data  Analysis 

In  order  to  extract  the  spectral  features  leading  to  further  analysis,  reduction  in  spectral  dimensionality  is 
applied  to  the  HSI  data  cube.  Principal  component  transform  is  used  to  de-correlate  data  and  maximize  the 
information  content  in  a  reduced  number  of  features  ^  Some  of  the  principal  components  calculated  from 
the  HSI  data  over  Dixie  are  shown  in  Figure  2.  Background  classes  of  open  area,  trees  and  roads  are 
apparent  in  the  first  and  third  components.  Fabric  nets  appear  with  strong  contrast  to  the  backgrounds  in 
the  seventh  component.  A  matched  filter  was  constructed  from  the  mean  of  several  pixels  extracted  from 
the  nets.  Matched  filtering  and  thresholding  are  then  applied  to  the  HSI  data  to  detect  all  pixels  of  fabric 


nets.  Figure  3  shows  the  net  detection.  Result  of  background  mapping  of  the  scene  is  also  included  in  the 
figure.  Roads,  grass,  trees  and  shadow  are  shown  as  separate  terrain  classes.  These  are  results  of  an 
unsupervised  data  clustering  operation  using  the  first  five  principal  components. 


HSI  RGB  Rendition  Background  (1st  PC) 


Road  (3rd  PC)  Camoufiage  Net  (7th  PC) 


Figure  2  Principal  components  calculated  from  the  HSI  of  Dixie  data  collect 
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Figure  3  Results  of  fabric  net  detection 
and  background  classification 
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2.2  Combined  SAR/HSI  Analysis  and  Fusion 

To  combine  SAR  and  HSI  detection  results,  co-registration  is  first  performed  with  reference  to  terrain 
features  such  as  open  and  tree  areas  using  scaling,  rotation  and  translation  operations.  Fusion  is  then 
carried  out  based  on  the  co-registered  images.  This  is  illustrated  in  Figure  4.  SAR  data  is  first  processed 
with  pixel  grouping  and  thresholding.  Target  detection  on  the  SAR  data  is  shown  on  the  left.  The  terrain 
map  derived  from  HSI  data  with  net  detection  is  depicted  in  the  middle  panel  of  Figure  4.  Combining  the 
analyses,  only  those  SAR  detections  located  in  either  open  area  or  around  fabric  nets  indicated  in  HSI  are 


retained.  SAR  detections  that  coincide  with  HSI  identifications  as  trees  and  are  far  from  open  areas  or  nets 
are  considered  false  alarms.  The  combined  detection  result  is  shown  on  the  right  of  the  figure.  A  SAR 
detection  of  vehicle  appears  under  a  net  at  the  top-right  corner.  Other  nets  are  empty  with  no  SAR 
detections.  There  are  several  strong  SAR  detections  at  the  left  side  of  the  open  area.  A  spectral  angle¬ 
mapping  algorithm  is  conducted  to  match  HSI  data  in  the  area  to  our  spectral  library  of  paints.  Three  pixels 
match  well  with  military  gray-tan  paint.  This  indicates  the  presence  of  a  possible  military  vehicle  and  thus 
confirms  the  SAR  detection. 
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Figure  4  Fusion  of  SAR  and  HSI  detections 


3  HSI/HPI  Fusion 

A  number  of  image  sharpening  approaches  have  been  attempted  and  applied  to  multi-spectral  images 
(MSI).  In  this  section,  three  approaches  are  briefly  reviewed  and  adapted  for  implementation  on  HSI. 

These  are  the  pseudo-inverse  technique  color  normalization  and  spatial  frequency  correction  methods. 
In  order  to  demonstrate  the  sharpening  approaches,  a  sample  data  is  created  based  on  field  data.  The 
algorithms  are  applied  to  the  generated  HSI  and  HPI  data  for  a  sharpened  HSI.  Algorithm  performance  is 
then  evaluated  by  comparing  with  the  original  truth  data.  To  exploit  the  full  utility  potential  of  combined 
HSI  and  HPI  data,  additional  spatial-spectral  analysis  is  also  shown  for  enhanced  detection  and 
identification. 

3. 1  Review  and  Demonstration  of  Sharpening  Approaches 

3.1.1  Pseudo-inverse  technique 

Given  a  high-resolution  panchromatic  image  (HPI)  co-registered  with  a  set  of  hyperspectral  images,  a 
system  of  equations  can  be  established  for  the  reconstruction  of  a  high  spatial  resolution  sharpened  HSI 
(SHSI).  The  value  at  a  pixel  in  HPI  is  the  spectral  sum  at  the  same  pixel  in  SHSI  and  the  spectral  value  at  a 
pixel  in  HSI  is  the  pixel-sum  over  the  HSI  GSD  in  the  same  spectral  band  of  SHSI.  If  the  sharpening  ratio 


is  r  and  the  number  of  spectral  bands  is  n,  then  the  number  of  equations  over  the  GSD  of  HSI  is  r^  +  n,  and 
the  number  of  unknowns  for  the  SHSI  reconstruction  is  r^  x  n.  As  the  sharpening  ratio  and  the  number  of 
spectral  band  increase,  the  number  of  unknowns  increases  faster  than  the  number  of  additional  equations. 
The  system  of  equations  is  generally  under  determined  for  a  unique  solution.  Pseudo  matrix  inversion 
algorithms  with  least-mean-squared  (LMS)  estimations  are  applied  to  obtain  a  solution  for  the  fusion 
problem.  The  method  is  described  in  the  following  equations: 

SPI  =  (A  aY  h,  A  PI  =  h 

Where  SPI  is  an  (n  .  r^)  x  1  array  containing  pseudo-inverse  sharpened  HSI  values  over  the  GSD  of  HSI, 

A  is  an  (n  +  r^)  x  (n  .  r^)  matrix  of  the  system  equation,  A^  (A  A^^  is  the  pseudo  inverse  of  A  and  h  is  an 
(n  +  r^)  X  1  array  of  the  HSI  and  HPI  input  values. 

In  this  approach,  it  is  important  that  the  panchromatic  image  is  in  perfect  registration  with  HSI.  The 
pseudo  inverse  technique  requires  singular  value  decomposition.  In  addition,  the  LMS  estimations  based 
on  the  under  determined  system  of  equations  may  not  achieve  the  required  spectral  accuracy. 

3. 1. 2  Color  Norm  alization 

The  color  normalization  algorithm  conventionally  used  in  MSl"^  is  modified  for  HSI  sharpening.  It 
multiplies  each  of  the  HSI  bands  by  the  HPI  and  the  resulting  values  are  each  normalized  by  the  averaged 
HPI  data  over  the  number  of  pixels  equivalent  to  the  GSD  of  HSI.  It  is  defined  by  the  following  equation: 

SCNi  =  (HSIi  *  HPI)  /  ave(HPI)GSD 

where  HSh  is  the  HSI  band  and  SCNi  is  the  sharpened  color  normalized  band. 

The  approach  is  straightforward  in  merging  the  spatial  contrast  of  HPI  into  spectral  bands  of  HSI.  Similar 
to  the  pseudo-inverse  technique  approach,  it  also  requires  precise  HSI/HPI  registration  and  the 
reconstructed  data  may  not  achieve  great  spectral  accuracy. 

3.1.3  Spatial  Frequency  Correction 

Some  sharpening  approaches  use  wavelet  transformations  Images  are  decomposed  into  different  spatial 
frequency  scales.  For  each  of  the  HSI  bands,  the  high  frequency  components  are  replaced  with  components 
from  HPI.  The  sharpened  image  is  then  obtained  via  inverse  transformation  of  the  modified  spectra.  In 
practice,  2-D  Fourier  transformation  can  be  applied  for  image  spatial  frequency  analysis.  The  sharpening 
process  is  described  as  below: 

SFCi  =  FFT  ‘  {FFT  (HSIOiow  +  FFT(HPI)hi} 

where  SFCi  is  sharpened  HSh  ,  FFT  (HSI^iow  represents  low  frequency  components  of  HSf  and  FFT(HPI)hi 
represents  high  frequency  components  of  HPI. 

In  this  approach,  the  spatial  shift  due  to  imperfect  HSI/HPI  registration  will  show  up  as  a  phase  shift  in  the 
frequency  spectrum.  In  addition,  careful  selection  of  the  number  of  high  frequency  components  for 
replacement  is  required  to  avoid  artifacts  in  the  sharpened  image.  Once  again,  some  loss  of  spectral  quality 
is  expected  in  the  reconstructed  data. 

3. 1. 4  Dem  onstration  Example 

In  the  three  sharpening  approaches  reviewed,  it  is  indicated  that  good  co-registration  between  HSI  and  HPI 
is  required.  Since  HSI  and  HPI  commonly  co-exist  on  the  same  platform,  co-registration  to  within  a  small 
fraction  of  an  HPI  pixel  is  expected.  However,  exploration  of  the  registration  issue  remains  an  important 
subject  in  the  ongoing  effort  for  sensor  fusion.  For  performance  comparisons,  perfect  co-registration  is 
assumed  here.  A  set  of  HSI  and  HPI  is  put  together  and  applied  with  the  sharpening  methods.  A  HYDICE 
(Hyperspectral  Digital  Image  Collection  Experiment)  major  frame  is  used  as  the  “truth”  in  both  spatial 
(0.8m/pixel)  and  spectral  (A^  -  10  nm)  domains.  This  input  “truth”  data  is  then  processed  to  generate  the 


contemporaneous  test  data  of  HSI  (4m/pixel,  AX  -  10  nm)  and  HPI  (0.8m/pixel,  broad  band).  The  first 
major  frame  of  HYDICE  data  from  Forest  Radiance  I  Run  05  is  used.  The  frame  is  a  portion  of  a  forest 
scene.  It  has  320x320  pixels,  0.8  m  per  pixel.  25  bands  from  0.63  |im  to  0.9  |im  are  used.  The  HPI  data  is 
generated  with  a  25-band  integration.  The  result  is  a  320x320  single  band  image,  0.8  m  per  pixel.  The 
HSI  data  is  obtained  by  under-sampling  the  spatially  blurred  input  data.  The  FWHM  (full  width  half 
maximum)  of  the  blurring  point  spread  function  (PSF)  is  5-pixel  wide  and  the  spatial  under-sampling  is  5  to 
1.  The  resulting  HSI  has  25  bands;  each  band  is  64x64  pixels  in  size  and  4m  per  pixel. 

Two  measures,  spectral  angle  and  spectral  distance  (Euclidean  distance),  are  used  to  evaluate  the 
sharpening  algorithms.  These  are  calculated  as: 

Spectral  angle  (A,  B)  =  COS  ‘(A  .  B  /  |A|  |B|) 

Spectral  distance  (A,  B)  =  |A  -  B| 

Where,  A  and  B  are  two  multi-band  spectra. 

A  zero  angle  or  zero  distance  represents  a  perfect  match  of  the  two  spectra.  The  sharpened  images  are 
compared  to  the  “truth”  data  in  terms  of  spectral  angle  and  spectral  distance.  The  spectral  distance 
normalized  by  the  pixel  amplitude  in  the  truth  image  is  calculated  for  the  fraction  of  spectral  difference. 

The  frame-averaged  differences  are  listed  in  Table  1.  For  comparison,  the  difference  measurements  from 
the  unsharpened  HSI  are  also  included  in  the  table.  It  shows  that  the  sharpened  images  improve 
significantly  from  the  unsharpened  HSI  in  spectral  distance,  but  show  no  obvious  improvement  in  spectral 
angle.  Among  the  sharpened  images,  color  normalization  (SCN)  is  closer  to  the  truth  in  spectral  angle  than 
results  of  pseudo-inverse  (SPI)  and  frequency  correction  (SFC).  On  the  other  hand,  the  image  of  frequency 
correction  (SFC)  is  the  closest  to  the  truth  in  spectral  distance.  Notice  that  the  comparisons  are  based  on 
averages  over  the  image.  The  sharpened  images  generally  appear  spatially  sharper.  However,  the  impact 
of  such  results  on  target  detection  and  identification  is  small.  Since  the  combined  HSI  and  HPI  data  is 
severely  under-determined,  spectral  features  of  objects  smaller  than  the  original  HSI  resolution  cannot  be 
fully  resolved  in  the  reconstruction  of  the  sharpened  HSI.  For  enhanced  target  detection  and  identification, 
additional  spatial  and  spectral  analysis  at  feature  level  is  necessary.  In  this  paper,  sharpened  HSI  image  by 
color  normalization  will  be  used  for  further  analysis. 

Table  1  Sharpening  Algorithm  Performance 


Average  spectral  angle  and  distance  over  major  frame  of  HYDICE  Forest  Radiance  I  Run  05 
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3.2  Spatial-spectral  Analysis 

Various  spatial  and  spectral  analysis  approaches  have  been  attempted  Here,  a  combined  spatial- 
spectral  analysis  is  carried  out  to  demonstrate  HSI  and  HPI  fusion  for  enhanced  background 
characterization  and  target  detection/identification.  An  analysis  approach  is  depicted  in  Figure  5.  The  HSI 
data  is  first  applied  with  background  classification  and  anomaly  detection.  The  background  class  statistics 
calculated  from  HSI  are  employed  when  the  sharpened  HSI  is  also  processed  with  background 
classification  and  target  detection.  At  the  same  time,  spatial  edge  detection  for  target  and  background 
boundaries  is  applied  to  the  HPI.  These  edges  are  combined  with  results  from  the  sharpened  HSI.  Targets 
and  backgrounds  are  spatially  defined  by  the  edges.  Subsequently,  their  materials  can  be  identified  with 
spectral  characteristics. 


In  order  to  evaluate  the  analysis  performance,  an  expanded  data  set  is  used  here.  Three  major  frames  of 
HYDICE  data  from  Forest  Radiance  I  Run  05  are  used  in  the  demonstration  example.  The  original  image 


of  300x960  pixels  in  size,  0.8  m  per  pixel  and  210  bands  (the  “truth”  image)  is  used  as  the  input.  The 
simulated  low-resolution  HSI  is  60x192  pixels  in  size,  4  m  per  pixel  and  210  bands.  The  HPI  is  obtained 
with  band  integration  from  0.4  to  0.8  |im,  300x960  pixels  with  0.8  m  per  pixel.  A  sharpened  HSI  data  cube 
is  constructed  from  the  HSI  and  HPI  with  the  color  normalization  method. 

Figure  6  shows  a  reference  image  (RGB)  of  HSI,  as  well  as  background  classification  and  anomaly 
detection  map  derived  from  the  unsharpened  HSI.  The  classification  is  performed  with  an  unsupervised 
feature  extraction  using  principal  component  analysis  (PCA)  The  Eigen  space  is  first  determined  for  the 
scene  from  the  covariance  matrix  of  the  data  cube.  It  then  divides  the  scene  into  classes  of  maximal 
separations  in  the  Eigen  space.  In  the  5-class  classification  shown  here,  road,  vegetation  (bright  and  dark), 
shade  and  ground  are  delineated  in  the  scene.  The  background  map  and  spectral  statistics  of  these  classes 
from  HSI  are  employed  in  all  subsequent  processing.  Anomalies  distinct  from  backgrounds  are  also 
detected.  These  detections  are  used  to  cue  for  target  detection  and  identification  in  the  combined  analysis. 
Edges  are  detected  from  HPI  with  a  SOBEL^^  operator.  Overlay  of  the  edges  with  background 
classification  from  sharpened  HSI  shows  better  defined  background  boundaries  than  in  the  unsharpened 
HSI.  Spectral  matched  filtering  is  applied  to  sharpened  HSI  data  at  regions  of  anomalies  from  the  earlier 
HSI  data  analysis.  Spectra  of  two  types  of  fabric  and  two  types  of  vehicle  paint  are  employed  in  the 
filtering.  The  results  of  fabric  and  vehicle  identification  are  shown  in  Figure  7.  Red  and  green  portions 
each  represent  the  detection  of  different  vehicle  paint.  The  detections  are  in  general  bounded  by  the  edges 
shown  in  blue.  An  enlarged  view  of  the  vehicle  detections  is  also  shown  in  the  figure.  The  vehicle  size 
and  orientation  can  be  determined  from  the  bounding  edges.  It  is  classified  as  large  (4x8  m^)  in  size  if  it  is 
4  to  6  pixels  wide  and  8  to  1 1  pixels  long  or  as  small  (3x6  m^)  in  size  if  it  is  2  to  5  pixels  wide  and  6  to  9 
pixels  long.  The  colored  bar  next  to  each  vehicle  in  the  enlarged  image  depicts  its  type  of  paint,  size  and 
orientation. 
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Figures  A  spatial-spectral  analysis  approach 
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Figure  6  RGB  image  ofHSI,  background  classification  and  anomaly  detection  derived  also  from  the 
unsharpened  HSI 
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Figure  7  Fabric  and  additional  object  identification 
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4  Summary 

HSI  offers  superior  spectral  resolution,  and  allows  background  characterization  and  material  identification. 
It  can  be  employed  in  conjunction  with  other  remote  sensing  data  to  further  enhance  the  overall  system 
performance.  In  this  paper,  the  potential  of  fusion  of  HSI  with  other  sensors  was  illustrated.  SAR 
compensated  a  disadvantage  of  HSI,  such  as  lack  of  surface  penetration  due  to  passive  sensing.  Their 
sensing  of  different  phenomenologies  also  helped  in  reduction  of  false  alarm  rates.  Another  shortcoming  of 
HSI,  due  to  tradeoff  with  rich  spectral  information,  is  sub-optimal  spatial  resolution.  This  can  also  be 
compensated  by  fusion  with  a  broadband  panchromatic  imager,  typically  with  much  better  spatial 
resolution. 

Framework  of  HSI  fusion  with  SAR  and  HPI  was  demonstrated.  A  common  data  set  of  P-3  SAR  and 
HYDICE  HSI  from  Dixie-97  data  collection  over  Vicksburg,  Mississippi  was  used  for  SAR/HSI  fusion. 
Targets  in  the  forest  background  included  fabric  nets  and  vehicles.  Principal  component  analysis  on  HSI 
data  was  shown  to  allow  effective  spectral  dimension  reduction  and  feature  extraction  for  terrain 
characterization  and  fabric  net  detection.  SAR/HSI  fusion  was  accomplished  with  a  co-registration  of  the 
images  using  references  to  terrain  features.  The  fusion  results  showed  detection  of  a  vehicle  under  a  fabric 
net  and  a  significant  reduction  of  SAR  false  alarms  due  to  trees.  In  addition,  a  case  of  SAR  detection  was 
confirmed  by  HSI  with  material  identification  of  military  vehicle  paint. 

For  HSI/HPI  data  fusion,  sharpening  approaches  were  investigated  and  implemented  on  a  combined  HSI 
and  HPI  data  set.  The  sharpened  HSI  retained  the  spectral  signatures  of  extended  area  in  HSI  and  in  general 
appeared  spatially  sharper.  However,  spectral  features  of  objects  smaller  than  the  original  HSI  resolution 
were  not  fully  resolved  through  sharpening  due  to  the  under-determined  nature  of  the  combined  data  set. 
Further  spatial  and  spectral  analysis  was  then  demonstrated  to  combine  high-resolution  information  from 
HSI  and  HPI  for  enhanced  background  characterization  and  target  detection  and  identification.  Anomalies 
distinct  from  backgrounds  were  also  detected  from  HSI  and  used  to  cue  for  target  detection  and 
identification  in  the  combined  analysis.  Spatial  image  processing  was  applied  to  HPI  for  edge  analysis.  As 
a  result  of  combined  analysis,  specific  target  material,  size  and  shape  were  determined. 
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